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Abstract Body 


Background / Context: 

American higher education has experienced a growing trend of globalization and 
internationalization during the past two decades. According to the Open Doors Report on 
International Educational Exchange, 260,327 students studied abroad for credit during academic 
year 2008-09, compared to fewer than 100,000 in 1996/1997 (Institute of International 
Education, 2010). A study abroad program is a summer, semester, or yearlong academic and/or 
practical experience overseas to enhance the participant’s formal education. The quality of and 
investment in study abroad programs are considered as an important indicator of overall 
institutional competitiveness. An institution that offers high-quality study abroad programs is 
perceived as likely to provide a stimulating learning environment for the students (Stroud, 2010). 
However, there are very few well-designed evaluation studies that provide evidence about the 
types of impacts of study abroad programs on students’ academic and non-academic outcomes. 
To fill this void in the literature, the present study aims to estimate, from an institutional 
perspective, the effect of study abroad experience on students’ social, behavioral, and altitudinal 
outcomes with an emphasis on students’ intercultural competence and adaptability using survey 
data of 1,330 college students who entered a northeastern mid-size comprehensive private 
university in the fall of 2006. The authors used propensity score matching to match treatment and 
comparison students on a wide range of baseline covariates so that the derived estimates of 
impacts would be as least biased as possible. 

Purpose / Objective / Research Question / Focus of Study: 

The Commission on the Abraham Eincoln Study Abroad Eellowship Program (2005) 
states that “moving overseas to study does not produce experts but it does begin a process of 
inculcating awareness of international and intercultural issues, a process that, multiplied many 
millions times over, promises to vastly increase American global literacy.” The hypotheses of the 
present study were guided by this statement. The author hypothesized that the study abroad 
experience is positively associated with habits of mind (e.g., critical thinking skills, problem- 
solving skills), social self-concept (e.g., tolerance of others with different beliefs, openness to 
having own views challenged), and civic-awareness (e.g., understanding of social problems 
facing the community, the nation and the world). The association of study abroad experience 
with students’ academic outcome is possible, but is not of main interest of this study. 

Setting: 

A mid-size comprehensive private university located in the Northeastern United States. 

Population / Participants / Subjects: 

The present study draws on the data of 1,004 college students’ responses to the Pall 2006 
Preshman Survey and Spring 2010 College Senior Survey, both of which were instruments 
developed by the Cooperative Institutional Research Program (CIRP) at UCLA. The CIRP 
Preshman Survey provides information about incoming students’ backgrounds, academic and 
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career expeetations, personal goals, and opinions on a wide range of politieal and soeial issues. 
The College Senior Survey eollects data on the students’ aeademie, civie, and diversity outcomes 
and post-eollege goals and plans. 

Table 1 lists the baseline survey items that refleet the students’ baekground information 
in the fall of 2006. The author merged the baseline survey data with the 2009-2010 College 
Senior Survey data, whieh resulted in an analytie sample of 1,004 students, ineluding 361 study- 
abroad students and 643 non-study-abroad students. While no one variable was missing more 
than 10% of the data, in eombination, listwise deletion ended getting rid of more than a third of 
the 1,004 eases. To prevent the huge loss of statistieal power, multiple imputations were 
eondueted to replaee all missing values on eaeh variable with multiple plausible values sampled 
from the predietion distribution of a regression model based on the observed data (Allison, 

2001). 

(please insert Table 1 here) 

Intervention / Program / Practice: 

The program or treatment variable in this study is the study abroad program that about 
one-third of the student sample partieipated in during their undergraduate years. The treatment 
group eonsists of the 361 students who studied abroad while the eomparison group eonsists of 
those who did not have any study-abroad experienee. 

Research Design: 

Although the experimental design is eonsidered as the most rigorous and robust method 
for program evaluation, it is infeasible and unethieal for this study beeause one eannot randomly 
assign some students to treatment and eomparison groups thereby providing potential benefits of 
the program to some students deprive learning opportunities of others. Therefore, the author 
turns to the non-equivalent eomparison group design that forms treatment and comparison 
groups via matching. Matching is a method of sampling from a large pool of potential eontrol 
units where the objeetive is to seleet a subset of the eontrol units with eovariate values similar to 
those in the treated group (Cook & Campbell, 1979). One ean attempt to mateh on all eovariates, 
but this may be difficult to implement when the set of eovariates is large (Coehran & Rubin, 
1973). In order to reduee the dimensionality of matching, Rosenbaum and Rubin (1983) 
proposed an innovative matehing technique that is based on the propensity seore p(X). This is 
defined as a single subjeet’s probability of reeeiving treatment given the eovariate values X and 
thus a sealar function of X. 

In this study, the author attempted to mateh students from the two groups based on their 
baseline eharaeteristies and henee reduce the seleetion biases due to their observed potential 
eonfounders. The author used the absolute value of the standardized differenee in means between 
treatment and eontrol groups introdueed by Rosenbaum and Rubin (1983) to examine bias 
reduetion before and after matehing. Aeeording to Rosenbaum and Rubin (1983), there is 
eovariate imbalanee and matehing is required if the absolute standardized differenee is greater 
than 10. Following a strategy suggested by Rosenbaum and Rubin (1984), the author built a 
propensity seore model using stepwise logistic regression with the seleeted eovariates. The 
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optimal matching is superior to greedy matehing in that it tends to minimize the average absolute 
propensity seore differenees between the matehed pairs and henee make the optimal global 
deeision (Gu & Rosenbaum, 1993). 


Data Collection and Analysis: 


Table 2 shows the means and standard deviations of the eovariates for the two groups 
before matehing. 


(please insert Table 2 here) 


Table 3 shows the results of balanee improvement on the eovariates after the optimal 
propensity seore matehing was eondueted. The matehing resulted in a subset of 361 eontrol 
students who matehed elosely with the 361 study-abroad students. As shown in Table 3, the 
average standardized differenee for all eovariates was redueed from 17.90 to 5.63. Additionally, 
the number of eovariates whose standardized differenee is above 10 has been redueed from 18 to 
6 as a result of the matehing. 


(please insert Table 3 here) 

It is noteworthy that 217 out of the 361 treatment students partieipated in the study 
abroad program for at least one fall or spring semester. The rest 144 treatment students only 
partieipated in the summer study abroad program, whieh is eonsidered substantially different 
from the fall/spring program. Therefore, we also eondueted a propensity seore matehing for the 
217 fall/spring study-abroad students versus the 643 potential eontrol students. Table 4 shows the 
eomparison of means and standard deviations of spring/fall study-abroad group versus non- 
study-abroad group before propensity seore matehing. Table 5 presents the means and standard 
deviations of study-abroad group versus non-study-abroad group after propensity seore 
matehing, whieh seleeted a subset of non-study-abroad students (N= 217) who matehed with the 
treatment group. As shown by Table 4 and 5, the average standardized differenee for all 
eovariates was redueed from 19.62 to 5.82. Additionally, the number of eovariates whose 
standardized differenee is above 10 has been redueed from 17 to 6 as a result of the matehing. 

(please insert Table 4 here) 

(please insert Table 5 here) 

Stepwise multiple regression analyses were eondueted using the matehed data to 
investigate whether study abroad experienee was assoeiated with students’ aeademie, civie, and 
behavioral outeomes after eontrolling for baseline individual and institutional eharaoteristies. All 
the independent variables that were retained in the multiple regression model have to meet the 
.15 signifieanee level. The outeome variables are the CIRP College Senior Survey eonstruets that 
eapture the students’ eurrieular and extra-eurrieular experienees and outeomes. The CIRP staff 
ereated these eonstruets by eombining individual items using the Item Response Theory (IRT). 
For example, the eonstruet Social Agency measures the extent to whieh students value politieal 
and soeial involvement as a personal goal. This eonstruet is eomprised of six items that assess 
the extent to whieh one engages in a eommunity aetion program, helping to promote raeial 
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understanding, becoming a community leader, influencing social values, keeping up to date with 
political affairs, and helping others who are in difficulty. 

Findings / Results: 

The results of post-matching multiple regression analysis show that, after controlling for 
baseline student characteristics, out of all the outcome constructs, study abroad in general 
(including the summer programs) is positively associated with change in habits of mind ip< 

.001) (e.g., critical thinking skills, problem-solving skills), social self-concept {p < .01) (e.g., 
tolerance of others with different beliefs, openness to having own views challenged), civic- 
awareness ip< .05) (e.g., understanding of social problems facing the community, the nation and 
the world), social-agency {p < .01) (e.g., participating in a community action program, helping to 
promote racial understanding). 

Conclusions: 

This study not only advances the understanding of the nature of impacts of study abroad 
programs, but also addresses the need for innovative methods to address the selection bias, i.e., 
individual and institutional factors that influence students’ intent to participate in the programs. 
Results from this study have important implications and show that the study abroad experience 
does have a positive impact on habits of mind, social self-concept, civic-awareness, and social- 
agency involvement, but no impacts on academic outcomes. The results confirmed the statement 
made by the Commission on the Abraham Lincoln Study Abroad Fellowship Program. In the full 
paper, the author will further explore if the duration of the programs moderate the impacts on 
students by looking at the impacts on the summer program participants and full-semester 
program participants. 
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Appendix B, Tables and Figures 

Table 1. The list of eovariates ehosen for the propensity seore matching. 


Variable Name 

Label 

SATV 

SAT Verbal 

SATM 

SAT Math 

female 

Gender (1= Female, 0= Male) 

white 

White 

hispanic 

Hispanic 

black 

Black 

income 

Household income 

engine 

Engineering major 

science 

Science major 

social 

Social science or humanities major 

health 

Health related major 

acaself 

TFS academic self-concept composite score 

socself 

TFS social self-concept composite score 

soagen 

TFS social agency composite score 

colrep 

TFS college reputation orientation composite score 

colinv 

TFS likelihood of college involvement composite score 

fstgen 

First generation college student 

fedu 

Father’s education (from 1 to 8; 1= grammar school, 8= graduate school) 

medu 

Mother’s education (from 1 to 8; 1= grammar school, 8= graduate school) 

catholic 

Catholic or not 

norelig 

No religion 

fincon 

Concern about ability to finance education (3= Most, 2= Some, 1= None) 
Proportion of white students in high school (5= completely white, 1= 

whiteness 

completely non-white) 

ap 

Number of AP courses taken in high school 

citizen 

U.S. citizen or not 

depend 

Number of dependents on parents for financial support 

hsgpa 

High school GPA 
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Table 2. The eomparison of means and standard deviations of study-abroad group versus non- 
study-abroad group before propensity score matching. 


Pre-Match 

Treatment (N- 

= 361) 

Comparison (N= 

643) 



Covariates 

Mean T SD 

T 

Mean C SD C 

Standardized Absolute 
Difference SD 

SATV 

632.79 

68.17 

614.08 

68.34 

27.42 

27.42 

SATM 

657.52 

63.95 

650.09 

67.97 

11.26 

11.26 

female 

0.64 

0.48 

0.46 

0.50 

37.30 

37.30 

white 

0.89 

0.31 

0.84 

0.37 

14.45 

14.45 

hispanic 

0.07 

0.25 

0.07 

0.26 

-3.19 

3.19 

black 

0.01 

0.12 

0.06 

0.24 

-26.04 

26.04 

income 

165029.82 

77158.38 147883.78 78635.80 

22.01 

22.01 

engine 

0.06 

0.24 

0.21 

0.41 

-43.13 

43.13 

science 

0.08 

0.27 

0.13 

0.33 

-16.05 

16.05 

social 

0.37 

0.48 

0.22 

0.42 

31.80 

31.80 

health 

0.02 

0.13 

0.08 

0.27 

-30.15 

30.15 

acaself 

52.61 

6.77 

52.58 

7.11 

0.43 

0.43 

socself 

50.20 

9.07 

49.76 

8.91 

4.86 

4.86 

soagen 

49.07 

7.98 

47.80 

8.50 

15.48 

15.48 

colrep 

51.61 

6.28 

51.51 

6.41 

1.58 

1.58 

colinv 

50.15 

6.49 

46.75 

6.90 

50.70 

50.70 

fstgen 

0.03 

0.16 

0.08 

0.27 

-21.97 

21.97 

fedu 

6.49 

1.50 

6.15 

1.68 

21.23 

21.23 

medu 

6.23 

1.44 

5.87 

1.61 

23.32 

23.32 

catholic 

0.70 

0.46 

0.71 

0.45 

-1.83 

1.83 

norelig 

0.08 

0.28 

0.07 

0.26 

3.73 

3.73 

fincon 

1.51 

0.60 

1.55 

0.64 

-6.12 

6.12 

whiteness 

3.81 

0.80 

3.72 

1.01 

10.15 

10.15 

ap 

3.52 

2.68 

2.88 

2.46 

24.71 

24.71 

citizen 

0.99 

0.10 

0.98 

0.15 

9.42 

9.42 

depend 

3.64 

1.42 

3.58 

1.52 

4.47 

4.47 

hsgpa 

3.61 

0.42 

3.52 

0.49 

20.44 

20.44 


Average 17.90 


* standardized difference = 




2 2 

e + V 

treated control 
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Table 3. The eomparison of means and standard deviations of study-abroad group versus non- 
study-abroad group after propensity seore matehing, whieh seleeted a subset of non-study-abroad 
students (N= 361) who matehed with the treatment group. 


Post-mateh 

Treatment (N= 

= 361) 

Comparison (N= 361) 



Covariates 

Mean T SD 

T 

Mean C ! 

Standardized Absolute 
SD C Differenee SD 

SATV 

632.79 

68.17 

624.48 

67.25 

12.28 

12.28 

SATM 

657.52 

63.95 

654.92 

65.07 

4.04 

4.04 

female 

0.64 

0.48 

0.55 

0.50 

19.23 

19.23 

white 

0.89 

0.31 

0.87 

0.33 

5.13 

5.13 

hispanie 

0.07 

0.25 

0.07 

0.25 

0.00 

0.00 

blaek 

0.01 

0.12 

0.02 

0.15 

-6.24 

6.24 

ineome 

165029.82 77158.38 

161901.96 

80599.29 

3.96 

3.96 

engine 

0.06 

0.24 

0.09 

0.29 

■10.36 

10.36 

seienee 

0.08 

0.27 

0.11 

0.31 

■11.38 

11.38 

soeial 

0.37 

0.48 

0.31 

0.46 

11.12 

11.12 

health 

0.02 

0.13 

0.02 

0.13 

0.00 

0.00 

aeaself 

52.61 

6.77 

52.46 

6.95 

2.07 

2.07 

soeself 

50.20 

9.07 

50.17 

8.86 

0.28 

0.28 

soagen 

49.07 

7.98 

48.94 

7.95 

1.74 

1.74 

eolrep 

51.61 

6.28 

51.64 

6.30 

-0.47 

0.47 

eolinv 

50.15 

6.49 

48.65 

6.05 

23.96 

23.96 

fstgen 

0.03 

0.16 

0.04 

0.19 

-4.73 

4.73 

fedu 

6.49 

1.50 

6.40 

1.56 

5.79 

5.79 

medu 

6.23 

1.44 

6.14 

1.47 

6.20 

6.20 

eatholie 

0.70 

0.46 

0.71 

0.46 

-1.21 

1.21 

norelig 

0.08 

0.28 

0.07 

0.26 

3.08 

3.08 

fineon 

1.51 

0.60 

1.52 

0.64 

-2.24 

2.24 

whiteness 

3.81 

0.80 

3.79 

0.94 

3.17 

3.17 

ap 

3.52 

2.68 

3.27 

2.62 

9.51 

9.51 

eitizen 

0.99 

0.10 

0.99 

0.10 

0.00 

0.00 

depend 

3.64 

1.42 

3.63 

1.45 

0.77 

0.77 

hsgpa 

3.61 

0.42 

3.60 

0.46 

3.13 

3.13 


Average 5.63 
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Table 4. The eomparison of means and standard deviations of fall/spring study-abroad group 
versus non-study-abroad group before propensity seore matehing. 


Pre-Mateh 

Treatment (N= 217) 

Comparison (N= 643) 



Covariates 

Mean T 

SD T 

Mean C 

SD C 

Standardized 

Differenee 

Absolute SD 

SATV 

635.63 

71.91 

614.08 

68.34 

30.72 

30.72 

SATM 

656.31 

66.97 

650.09 

67.97 

9.21 

9.21 

female 

0.65 

0.48 

0.46 

0.50 

40.42 

40.42 

white 

0.85 

0.36 

0.84 

0.37 

3.52 

3.52 

hispanie 

0.08 

0.27 

0.07 

0.26 

1.39 

1.39 

blaek 

0.02 

0.15 

0.06 

0.24 

-20.06 

20.06 

ineome 

155818.11 

78924.12 147883.78 

78635.80 

10.07 

10.07 

engine 

0.04 

0.19 

0.21 

0.41 

-54.12 

54.12 

seienee 

0.05 

0.21 

0.13 

0.33 

-28.75 

28.75 

soeial 

0.43 

0.50 

0.22 

0.42 

46.00 

46.00 

health 

0.00 

0.00 

0.08 

0.27 

-41.92 

41.92 

aeaself 

52.33 

6.69 

52.58 

7.11 

-3.55 

3.55 

soeself 

49.88 

8.63 

49.76 

8.91 

1.34 

1.34 

soagen 

49.59 

8.16 

47.80 

8.50 

21.57 

21.57 

eolrep 

51.30 

6.38 

51.51 

6.41 

-3.24 

3.24 

eolinv 

50.89 

6.71 

46.75 

6.90 

60.72 

60.72 

fstgen 

0.03 

0.18 

0.08 

0.27 

-19.47 

19.47 

fedu 

6.45 

1.53 

6.15 

1.68 

18.61 

18.61 

medu 

6.20 

1.40 

5.87 

1.61 

22.00 

22.00 

eatholie 

0.66 

0.48 

0.71 

0.45 

-10.79 

10.79 

norelig 

0.10 

0.30 

0.07 

0.26 

10.02 

10.02 

fineon 

1.54 

0.61 

1.55 

0.64 

-0.59 

0.59 

whiteness 

3.81 

0.77 

3.72 

1.01 

9.94 

9.94 

ap 

3.74 

2.69 

2.88 

2.46 

33.27 

33.27 

eitizen 

0.98 

0.13 

0.98 

0.15 

3.42 

3.42 

depend 

3.62 

1.46 

3.58 

1.52 

2.72 

2.72 

hsgpa 

3.62 

0.42 

3.52 

0.49 

22.32 

22.32 


Average 19.62 


SREE Spring 2013 Conference Abstract Template 


B-4 



Table 5. The eomparison of means and standard deviations of fall/spring study-abroad group 
versus non-study-abroad group after propensity seore matehing. 


Post-mateh 

Treatment (N= 

217) 

Comparison (N= 

= 217) 



Covariates 

Mean T SD 

T 

Mean C SD 

C 

Standardized 

Differenee 

Absolute SD 

SATV 

635.63 

71.91 

625.76 

69.94 

13.92 

13.92 

SATM 

656.31 

66.97 

648.15 

64.65 

12.40 

12.40 

female 

0.65 

0.48 

0.61 

0.49 

8.60 

8.60 

white 

0.85 

0.36 

0.84 

0.37 

3.82 

3.82 

hispanie 

0.08 

0.27 

0.09 

0.29 

-4.94 

4.94 

blaek 

0.02 

0.15 

0.04 

0.20 

-10.42 

10.42 

ineome 

155818.11 78924.12 

151799.63 80575.70 

5.04 

5.04 

engine 

0.04 

0.19 

0.04 

0.20 

-2.37 

2.37 

seienee 

0.05 

0.21 

0.04 

0.20 

2.25 

2.25 

soeial 

0.43 

0.50 

0.41 

0.49 

4.66 

4.66 

health 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

aeaself 

52.33 

6.69 

52.19 

6.89 

2.11 

2.11 

soeself 

49.88 

8.63 

50.69 

9.19 

-9.06 

9.06 

soagen 

49.59 

8.16 

49.83 

7.95 

-2.99 

2.99 

eolrep 

51.30 

6.38 

51.51 

6.67 

-3.29 

3.29 

eolinv 

50.89 

6.71 

50.16 

6.28 

11.22 

11.22 

fstgen 

0.03 

0.18 

0.05 

0.21 

-7.11 

7.11 

fedu 

6.45 

1.53 

6.29 

1.54 

10.82 

10.82 

medu 

6.20 

1.40 

6.11 

1.49 

6.21 

6.21 

eatholie 

0.66 

0.48 

0.67 

0.47 

-1.95 

1.95 

norelig 

0.10 

0.30 

0.09 

0.28 

4.72 

4.72 

fineon 

1.54 

0.61 

1.57 

0.67 

-4.32 

4.32 

whiteness 

3.81 

0.77 

3.80 

0.89 

1.66 

1.66 

ap 

3.74 

2.69 

3.72 

2.84 

0.67 

0.67 

eitizen 

0.98 

0.13 

0.99 

0.12 

-3.65 

3.65 

depend 

3.62 

1.46 

3.55 

1.42 

4.49 

4.49 

hsgpa 

3.62 

0.42 

3.55 

0.48 

14.34 

14.34 


Average 5.82 
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